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DETAILED ACTION 

1 . This action is responsive to communications: RCE and IDS filed on 6/28/03 to 
the application filed on 1 1/20/98, priority filed 12/2/97. 

2. Claims 1-22 are pending in the case. Claims 1, 10, 14, 18 and 22 are 
independent claims. 

Priority 

3. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)»(d), which 
papers have been placed of record in the file. 

Claim Rejections - 35 (JSC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 
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6. Claims 1-21 remain rejected under 35 U.S.C. 103(a) as being unpatentable over 

Sato (JP-8-293039A, 1 1/5/96, filed 4/24/95) in view of Ohba (JP-3-216767A, 9/21/91 , 

filed 1/21/90), the IDSs submitted by Applicants. 

Regarding independent claim 1, Sato discloses: 

- sequentially providing music control information in correspondence with the 
music to be played, the music control information including a plurality of types of 
MIDI event data associated with the music to be played (abstract: "it is possible 
to generate from music, images having definite shape and motion, such as the 
motion of a human dancing to the music"; [0002]: "the motion of sounds in each 
part of a music is made to correspond to the motion of images having basic 
shape"; [0006]: "musical information is converted into a series of pieces of motion 
information by appropriately switching the motion at every point of sound 
buildup", "motions of the articulated object are applied to the image ....to display 
an animated image"; [0014]: the motions of limbs of the articulated object are 
stored in advance in the motion database and the chords are detected to match 
with the motion data stored in the motion database] [0033]: ".. As music 
information, signals corresponding to performance control information, such as 
MIDI control signals are inputted to the musical signal inputting section 101. The 
MIDI control signals have interval information and sound volume information for 
individual tones"; [0035]: "it is possible to convert the acoustic art, the music, into 
specific visual art, the motion of articulated object having a definite shape, for 
example the motion of a human dancing to the music") 



Application/Control Number: 09/1 97,1 84 Page 4 

Art Unit: 2178 

- generating a sound in accordance with the music control information to thereby 
play the music ([0006]: a sound buildup point in accordance with the musical 
information and the motions of the animated image) 

- generating a motion image of the object in matching with progression of the 
music ([0014]: the motions of limbs of the articulated object is matched with the 
chord; [0015]: "an articulated object moving in time to the music may be 
displayed using the pre-stored motion data") 

Sato does not disclose explicitly: 

- selecting and setting the types of MIDI event data to the movable parts of the 
object such that the respective movable parts correspond to the types of MIDI 
event data 

- controlling the movements of the respective movable parts in correspondence to 
the types of MIDI event data included in the music control information 
sequentially provided, thereby generating a motion image of the object in 
matching with progression of the music 

Ohba discloses: 

- selecting and setting the types of MIDI event data to the movable parts of the 
object such that the respective movable parts in correspondence to the types of 
MIDI event data (pages 1 and 2, as shown in figures 2 and 3: selecting and 
setting the motions of the legs and the arms, which are movable parts of an 
object, such that the respective movable parts in correspondence to the types of 
event data, e.g. right leg or left leg moving forward or backward, right arm or left 
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arm swinging forward or backward; pages 3 and 4: the different parameters RQ, 
RR and RS generated that designate the transformation of input images 
according to the MIDI signals indicate that the respective movable parts of the 
object is set for the corresponding types of MIDI event data since each 
parameter is assigned for each movement of legs or arms and for each MIDI 
signal to apply for selecting purpose) 
- controlling the movements of respective movable parts in correspondence to the 
types of MIDI event data included in the music control in correspondence to the 
types of MIDI event data contained in the music control information sequentially 
provided (page 2: the motions of legs and the motions of arms are generated 
with correspondent parameters; page 3: the shape of the arm of the character is 
transformed so that the character may move the arms; the shapes of the legs 
and the arms are synthesized to form the entire configuration of character for 
outputting the animation; a filter is formed to detect musical scale, stress and 
length of the synthesized sound based on the MIDI signal to generate the 
parameters RQ, RR, and RS correspondent to the change of the arms and legs 
of the animated object according to the music) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato since Ohba provides the movements of 
respective movable parts of an animated objects according to the MIDI signals and Sato 
provides the music control information for the movable parts of the objects to display the 
movement of the body of a human to the music. The combination of Ohba and Sato 
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would enhance the animation display with music by applying the control of the 
movements of an animated object so that said movements are performed along with the 
music. 

Regarding claim 2, which is dependent on claim 1, Sato discloses preparing a frame of 
the motion image in advance to generation of the sound corresponding to the same 
data block to generate the prepared frame timely when generating the sound according 
to the same data block used for preparation of the frame (section [0014], ...chords 
detected with the chord detecting section are matched in advance with the motion data 
stored in the motion database..., motion output section visualizes to display the motion 
data generated...). 

Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 

Regarding claim 3, which is dependent on claim 1 , Sato discloses generating the key 
frames of the motion image in response to the music control information, and generating 
the sub frames inserted between the successive key frames by interpolation to 
smoothen the motion image ([0027]: generating M frames per one motion motion 



Application/Control Number: 09/197,184 Page 7 

Art Unit: 2178 

data for display are obtained by interpolating a motion data of m-th frame and a motion 
data of (m+1)th frame... generate motions according to the music) 

Regarding claim 4, which is dependent on claim 1, Sato does not disclose generating 
the motion image of an object representing an instrument player and analyzing the 
audio control information to determine a rendition movement of the instrument player for 
controlling the motion image as if the instrument player plays the music. 
Instead Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; sections [0033] and [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 

Regarding claim 5, which is dependent on claim 1, Sato does not disclose determining 
the movement of the movable parts of the object, generating the motion image 
according to the motion parameters, and resetting the motion image to revert the 
movable parts to the default positions in matching with the progression of the music. 
Instead Sato discloses the matching of motions of a human dancing to the music 
(abstract; [0005] and [0035]). 
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Ohba discloses generating the motion image according to the motion parameters 
(pages 1 and 2, parameter Q associated with the motions of legs, parameter R 
associated with the motions of arms, parameter S associated with the display position of 
the character... transform the shapes of respective parts of the character expressed with 
the shape data in response to the values of parameters Q, R, S). 
Ohba does not disclose determining the movement of the movable parts of the object. 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Ohba to include "determining the movement ..." 
for the following reason. "Generating the motion image ... in Ohba inherently includes 
"determining the movement..." since the movements of movable parts of an object 
should be determined to generate a motion image of the object. 
Also, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have combined Ohba into Sata for facilitating the control of 
motions of an objects in accordance to the played music by using the set parameters for 
corresponding motions. 

Regarding claim 6, which is dependent on claim 1 , Sato discloses that the 
synchronization signal, which is provided from the sequencer module and which is 
utilized to regulate a beat of music so that the motion image of the object is controlled in 
synchronization with the beat of the music (sections [0008] - [001 1]). 
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Regarding claim 7, which is dependent on claim 1, Sato discloses the synchronization 
of the motion video and the music (abstract, sections [0027]-[0029]). However, Sato 
does not disclose providing the music control information containing event data 
specifying an instrument used to play the music. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to add specifying an instrument used to play the music 
since it was well known that a music must be played by an instrument or a combination 
of some instruments, and since the music control information is where to include 
information relating to playing the music. 

Regarding claim 8, which is dependent on claim 1, Sato does not disclose setting 
motion parameters effective to determine the movements of the movable parts of the 
object 

Ohba discloses setting motion parameters effective to determine the movements of the 
movable parts of the object (page 2: parameter Q is stored for designating the motions 
of legs, parameter R is stored for designating the motions of arms, ...) 
Ohba also discloses utilizing the motion parameters to control the motion image of the 
object (refer to claim 1 ). 

Sato and Ohba do not disclose controlling the amplitude of the sound to further control 
the motion image such that the movement of each part determined by the motion 
parameter is scaled in association with the amplitude of the sound. 
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amplitude of the sound to further control the motion image such that the movement of 
each part determined by the motion parameter is scaled in association with the 
amplitude of the sound because of the following reason. The amplitude of the sound 
can be controlled by adjusting the volume of the sound. Since the movement of the part 
of the object is determined by a motion parameter, said movement is synchronized with 
the music, and the amplitude of the sound can be adjusted, these features suggest to 
synchronize the movement of the motion images with the sound in a way that the 
motion images are adjusted to match with the up and down of the sound. In other 
words, the motion images are scaled in association with the amplitude of the sound. 
Regarding claim 9, which is dependent on claim 1, as mentioned in claim 6, Sato 
discloses that the dancer object, which is a motion object, and the music are 
synchronized (abstract, [0035]). Ohba discloses the motion parameters correspondent 
to the movements of a motion object (pages 1-2). 

Sato does not disclose explicitly that the synchronization signal is provided from the 
sequencer module. However, it would have been obvious to one of ordinary skill in the 
art at the time of the invention was made to have modified Sato to include a sequencer 
module since Sato's disclosure of synchronizing the motions of a dancer object to the 
music suggests such module in Sato for performing the function. 

Claims 10-13 are for an apparatus of claims 1-4, and therefore are rejected under the 
same rationale. 
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Regarding independent claim 14, Sato discloses: 

- sequentially providing performance data to perform the music, the performance 
data including a plurality of types of MIDI event data associated with the music to 
be played (abstract, possible to generate from music the motion of a human 
dancing to the music; section [0002], each part of music is made to correspond to 
the motion of images having basic shape; section [0006], musical information 
and motions of the articulated object are applied to the image ....to display an 
animated image; section [0008]; section [0035], motion of a human dancing to 
the music) 

- generating a sound in accordance with the performance data to thereby perform 
the music (section [0006], a sound buildup point in accordance with the musical 
information and the motions of the animated image) 

- generating a motion image of the object in matching with the progression of the 
music, wherein the step of generating a motion image is in response to the 
performance data for controlling movements of the respective movable parts in 
correspondence to the types of MIDI event data included in the performance data 
sequentially provided by said step of sequentially providing performance data 
(sections [0014] and [0015], the motions of limbs of the articulated object are 
matched with the chord; section [0005], generating timing to switch the motion of 
the articulated object from musical signal, detecting chord from musical signals, 
and selecting motions from motion database based on the chords in time to the 
timing to switch the motion) 
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Sato does not disclose explicitly selecting and setting the types of MIDI event data to 
the movable parts of the object such that the respective movable parts correspond to 
the types of the MIDI event data selected and set. 

Ohba discloses selecting and setting the types of MIDI event data to the movable parts 
of the object such that the respective movable parts correspond to the types of the MIDI 
event data selected and set (pages 1 and 2, Summary of the Invention and 
Embodiment: producing an animation of the object accordingly to the MIDI signal, and 
the motion of legs and the motion of arms show the movable parts of the objects 
according to the MIDI signal; page 4: the fact that the parameters RQ, RR, and RS 
generated to designate the transformation of input images according to the MIDI signals 
indicates that the types of MIDI data to the movable parts - the legs and the arms - are 
set and selected via the parameters showing the movement of the parts of the object). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato to facilitate reproducing a motion object 
where the movement of the object is synchronized with the sound of the music via 
selecting the set MIDI event data to the movable parts of the objects to obtain a desired 
output where MIDI signals are synchronized with the transformation of the input image, 
which is the movement of the parts of the object. 

Regarding claim 15, which is dependent on claim 14, Sato discloses: 

- generating a sound in response to the timing signal and in accordance with the 
performance data to thereby perform the music (section [0006], a sound buildup 
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point in accordance with the musical information and the motions of the animated 
image) 

- generating a motion image of the object to response to the timing signal to 
matching with the progression of the music (section [0005], generating timing to 
switch the motion of the articulated object from musical signal, detecting chord 
from musical signals, and selecting motions from motion database based on the 
chords in time to the timing to switch the motion) 

- preparing a frame of the motion image in advance to generation of the sound 
corresponding to the motion image so that the prepared frame can be generated 
timely when the sound is generated according to the same block used for 
preparation of the frame (section [0014], motions of limbs of the articulated object 
are stored in advance in the motion database, and the kinds of chords detected 
are matched in advance with the motion data stored in the motion database) 

Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 

Regarding claim 16, which is dependent on claim 14, Sato discloses generating the key 
frames of the motion image in response to the synchronization signal according to the 
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motion parameters and the music control information, and generating the sub frames 
inserted between the successive key frames by interpolation to smoothen the motion 

image (section [0027], generating M frames per one motion motion data for display 

are obtained by interpolating a motion data of m-th frame and a motion data of (m+1 )th 
frame... generate motions according to the music). 

Regarding claim 17, which is dependent on claim 14, as mentioned in claim 14, Sato 
does not disclose providing motion parameters to design a movement of the object 
representing a player of an instrument, and wherein the step of generating a motion 
image comprises utilizing the motion parameters to form the framework of the motion 
image of the player and utilizing the performance data to modify the framework for 
generating the motion image presenting the player playing the instrument to perform the 
music. 

Instead, Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; [0033], [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 
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Claims 18-21 are for a machine readable medium of claims 1-4, and therefore are 
rejected under the same rationale. 

7. Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Sato 

(JP-8-293039A, 11/5/96, filed 4/24/95) in view of Ohba (JP-3-216767A, 9/21/91, filed 

1/21/90), the IDSs submitted by Applicants. 

Regarding independent claim 22, Sato discloses: 

- providing music control information in correspondence with the music to be 
played such that the music control information is arranged into a plurality of 
channels (abstract: "it is possible to generate from music, images having definite 
shape and motion, such as the motion of a human dancing to the music"; [0002]: 
"the motion of sounds in each part of a music is made to correspond to the 
motion of images having basic shape"; [0006]: "musical information is converted 
into a series of pieces of motion information by appropriately switching the 
motion at every point of sound buildup", "motions of the articulated object are 
applied to the image ....to display an animated image"; [0014]: the motions of 
limbs of the articulated object are stored in advance in the motion database and 
the chords are detected to match with the motion data stored in the motion 
database] [0035]: "it is possible to convert the acoustic art, the music, into 
specific visual art, the motion of articulated object having a definite shape, for 
example the motion of a human dancing to the music") 
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- generating a sound in accordance with the music control information to thereby 
play the music ([0006]: a sound buildup point in accordance with the musical 
information and the motions of the animated image) 

- generating a motion image of the object in matching with progression of the 
music ([0014]: the motions of limbs of the articulated object is matched with the 
chord; [0015]: "an articulated object moving in time to the music may be 
displayed using the pre-stored motion data") 

- controlling movements of the respective movable parts in correspondence to the 
channels of the music control information sequentially provided from the 
sequencer module, thereby generating a motion image of the object in matching 
with progression of the music 

Sato does not disclose: 

- selecting and setting the channels of the music control information to the 
movable parts of the objects such that the respective movable parts correspond 
to the channels of the music control information 

- controlling movements of the respective movable parts in correspondence to the 
channels of the music control information sequentially provided from the 
sequencer module 

Ohba discloses: 

- selecting and setting the types of event data to the movable parts of the object 
such that the respective movable parts in correspondence to the types of event 
data (pages 1 and 2, as shown in figures 2 and 3: selecting and setting the 
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motions of the legs and the arms, which are movable parts of an object, such that 
the respective movable parts in correspondence to the types of event data, e.g. 
right leg or left leg moving forward or backward, right arm or left arm swinging 
forward or backward) 
- controlling the movements of respective movable parts in correspondence to the 
types of event data contained in the music control in correspondence to the types 
of event data contained in the music control information sequentially provided 
(page 2: the motions of legs and the motions of arms are generated with 
correspondent parameters; page 3: the shape of the arm of the character is 
transformed so that the character may move the arms; the shapes of the legs 
and the arms are synthesized to form the entire configuration of character for 
outputting the animation; a filter is formed to detect musical scale, stress and 
length of the synthesized sound based on the MIDI signal to generate the 
parameters RQ, RR, and RS correspondent to the change of the arms and legs 
of the animated object according to the music) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato to enhance the animation display by 
applying the control of the movements of an animated object so that said movements 
are performed along with the music. 
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Response to Arguments 



8. Applicant's arguments filed 6/28/03 have been fully considered but they are not 
persuasive. 

During the phone interview, the representative of the Applicants argued that the event 
data in the claims are the MIDI event data. Examiner did not agree since the MIDI 
event data is not claimed. Therefore, the event data can be interpreted as any event 
data. 

As amended, claims 1, 10, 14, 18 now are clarified that the "event data" is the "MIDI 
event data." 

However, during reconsideration, the Examiner sees that both Sato and Ohba apply 
MIDI signals for controlling the synchronization of music and the movement of the parts 
of the object (Sato, pages 2-6; Ohba, pages 1-4). The MIDI signals in Sato and Ohba 
inherently indicates that the event data should be the MIDI event data. Sato and Ohba, 
therefore, suggest the claimed feature. 

Applicants also argue that neither Sato nor Ohba contain any disclosure or suggestion 
of the parameter setting module as recited in the amended claims (Remarks, page 6). 
Examiner respectfully disagrees. 

The parameter setting module, as claimed, is for selecting and setting the types of MIDI 
event data to the movable parts of the object such that the respective movable parts 
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correspond to the types of MIDI event data selected and set by the parameter setting 
module. 

Ohba discloses selecting and setting the types of event data to the movable parts of the 
object such that the respective movable parts in correspondence to the types of MIDI 
event data ( pages 1 and 2, as shown in figures 2 and 3 : selecting and setting the 
motions of the legs and the arms, which are movable parts of an object, such that the 
respective movable parts in correspondence to the types of event data, e.g. right leg or 
left leg moving forward or backward, right arm or left arm swinging forward or backward; 
pages 3 and 4 : the different parameters RQ, RR and RS generated that designate the 
transformation of input images according to the MIDI signals indicate that the respective 
movable parts of the object is set for the corresponding types of MIDI event data since 
each parameter is assigned for each movement of legs or arms and for each MIDI 
signal to apply for selecting purpose later). 

Regarding independent claim 22, Applicants argue that none of the cited references, 
including Sato and Ohba, contain any disclosure or suggestion of a parameter setting 
module that is operable to select and set the channels of music control information to 
the movable parts of an animated object. 
Examiner respectfully disagrees. 

As mentioned above, Ohba suggests selecting and setting music control information to 
the movable parts of an animated object. 
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Conclusion 



9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Luther (US Pat No. 5,640,590, 6/17/97). 

Goede (US Pat No. 5,952,598, 9/14/99, filed 9/10/97, priority 6/7/96). 
Tada (US Pat No. 5,915,972, 6/29/99). 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 703-305- 
0432. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached on 703-308-5186. The fax phone 
numbers for the organization where this application or proceeding is assigned are 703- 
746-7239 for regular communications and 707-746-7238 for After Final 
communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
9000. , 



clh 

8/30/03 




